Gaussian Processes to Speed up Hybrid Monte Carlo for Expensive Bayesian Integrals
نویسنده
چکیده
Hybrid Monte Carlo (HMC) is often the method of choice for computing Bayesian integrals that are not analytically tractable. However the success of this method may require a very large number of evaluations of the (un-normalized) posterior and its partial derivatives. In situations where the posterior is computationally costly to evaluate, this may lead to an unacceptable computational load for HMC. I propose to use a Gaussian Process model of the (log of the) posterior for most of the computations required by HMC. Within this scheme only occasional evaluation of the actual posterior is required to guarantee that the samples generated have exactly the desired distribution, even if the GP model is somewhat inaccurate. The method is demonstrated on a 10 dimensional problem, where 200 evaluations suffice for the generation of 100 roughly independent points from the posterior. Thus, the proposed scheme allows Bayesian treatment of models with posteriors that are computationally demanding, such as models involving computer simulation.
منابع مشابه
Gaussian Processes for Bayesian Classification via Hybrid Monte Carlo
The full Bayesian method for applying neural networks to a prediction problem is to set up the prior/hyperprior structure for the net and then perform the necessary integrals. However, these integrals are not tractable analytically, and Markov Chain Monte Carlo (MCMC) methods are slow, especially if the parameter space is high-dimensional. Using Gaussian processes we can approximate the weight ...
متن کاملGaussian Processes for Bayesian Classiication via Hybrid Monte Carlo
The full Bayesian method for applying neural networks to a prediction problem is to set up the prior/hyperprior structure for the net and then perform the necessary integrals. However, these inte-grals are not tractable analytically, and Markov Chain Monte Carlo (MCMC) methods are slow, especially if the parameter space is high-dimensional. Using Gaussian processes we can approximate the weight...
متن کاملExact Hamiltonian Monte Carlo for Truncated Multivariate Gaussians
We present a Hamiltonian Monte Carlo algorithm to sample from multivariate Gaussian distributions in which the target space is constrained by linear and quadratic inequalities or products thereof. The Hamiltonian equations of motion can be integrated exactly and there are no parameters to tune. The algorithm mixes faster and is more efficient than Gibbs sampling. The runtime depends on the numb...
متن کاملApproximating Bayesian Posterior Means Using Multivariate Gaussian Quadrature
Multiple integrals encountered when evaluating posterior densities in Bayesian estimation were approximated using Multivariate Gaussian Quadrature (MGQ). Experimental results for a linear regression model suggest MGQ provides better approximations to unknown parameters and error variance than simple Monte Carlo based approximations. The authors are Graduate Research Assistant, Professor and Gra...
متن کامل